Validation Pipeline
- 38 installs
- 433 repo stars
- Updated August 4, 2026
- proffesor-for-testing/agentic-qe
validation-pipeline is a Claude Code skill for ai & agent building.
About
validation-pipeline is a Claude Code skill for ai & agent building. It helps solo builders move faster with AI-assisted development.
- validation-pipeline
- AI & Agent Building
- AI-coding skill
Validation Pipeline by the numbers
- 38 all-time installs (skills.sh)
- +3 installs in the week ending Aug 4, 2026 (Skillselion tracking)
- Ranked #8,404 of 16,546 AI & Agent Building skills by installs in the Skillselion catalog
- Data as of Aug 5, 2026 (Skillselion catalog sync)
npx skills add https://github.com/proffesor-for-testing/agentic-qe --skill validation-pipelineAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 38 |
|---|---|
| repo stars | ★ 433 |
| Last updated | August 4, 2026 |
| Repository | proffesor-for-testing/agentic-qe ↗ |
How do I helps with ai & agent building tasks.?
Helps with ai & agent building tasks.
Who is it for?
Best when you're working on ai & agent building and need structured help with validation pipeline.
Skip if: Teams with no ai & agent building needs, or anyone wanting a generic chat assistant without this specific workflow.
When should I use this skill?
When you need to helps with ai & agent building tasks., or when validation-pipeline is a claude code skill for ai & agent building.
What you get
Structured output aligned to validation-pipeline: validation-pipeline, AI & Agent Building.
Files
Validation Pipeline
Purpose
Run structured validation pipelines that execute steps sequentially, enforce gates at blocking failures, and produce scored reports. Uses .claude/helpers/validation-pipeline.cjs with 13 requirements validation steps (BMAD-003).
Activation
- When validating requirements documents
- When running structured quality gates
- When assessing document completeness, testability, or traceability
- When invoked via
/validation-pipeline
Quick Start
# Validate a requirements document (all 13 steps)
/validation-pipeline requirements docs/requirements.md
# Validate with specific steps only
/validation-pipeline requirements docs/requirements.md --steps format-check,completeness-check,invest-criteria
# Continue past blocking failures
/validation-pipeline requirements docs/requirements.md --continue-on-failure
# Output as JSON
/validation-pipeline requirements docs/requirements.md --jsonWorkflow
Step 1: Read the Target Document
Read the file specified by the user. If no file is provided, ask for one.
Read the target document using the Read tool.
Store the content for pipeline execution.Step 2: Select Pipeline
Choose the appropriate pipeline based on the user's request:
| Pipeline | Steps | Use Case |
|---|---|---|
requirements | 13 | Requirements documents, PRDs, user stories |
Additional pipelines can be added to .claude/helpers/validation-pipeline.cjs.
Step 3: Execute Pipeline
The pipeline helper (.claude/helpers/validation-pipeline.cjs) handles execution:
1. Sequential execution — steps run in order, each receiving results from prior steps 2. Gate enforcement — blocking steps that fail halt the pipeline (unless --continue-on-failure) 3. Per-step scoring — each step produces a 0-100 score with findings and evidence 4. Weighted rollup — overall score uses category weights (format=10%, content=30%, quality=25%, traceability=20%, compliance=15%)
Requirements Pipeline Steps (13 total)
| # | Step ID | Category | Severity | What It Checks |
|---|---|---|---|---|
| 1 | format-check | format | blocking | Headings, required sections, document length |
| 2 | completeness-check | content | blocking | Required fields populated, acceptance criteria present |
| 3 | invest-criteria | quality | warning | Independent, Negotiable, Valuable, Estimable, Small, Testable |
| 4 | smart-acceptance | quality | warning | Specific, Measurable, Achievable, Relevant, Time-bound |
| 5 | testability-score | quality | warning | Can each requirement be tested? |
| 6 | vague-term-detection | content | info | Flags "should", "might", "various", "etc." |
| 7 | information-density | content | info | Every sentence carries weight, no filler |
| 8 | traceability-check | traceability | warning | Requirements-to-tests mapping exists |
| 9 | implementation-leakage | quality | warning | Requirements don't prescribe implementation |
| 10 | domain-compliance | compliance | info | Alignment with domain model |
| 11 | dependency-analysis | traceability | info | Cross-requirement dependencies identified |
| 12 | bdd-scenario-generation | quality | warning | Can generate Given/When/Then for each requirement |
| 13 | holistic-quality | compliance | blocking | Overall coherence, no contradictions |
Step 4: Report Results
Format the pipeline result as a structured report:
# Validation Report: Requirements Pipeline
**Overall**: PASS/FAIL/WARN | **Score**: 85/100 | **Duration**: 42ms
## Step Results
| # | Step | Status | Score | Findings | Duration |
|---|------|--------|-------|----------|----------|
| 1 | Format Check | PASS | 100 | 0 | 2ms |
| 2 | Completeness | WARN | 60 | 2 | 5ms |
...
## Blockers
- (blocking findings listed here)
## All Findings
- [HIGH] Missing acceptance criteria: Requirement US-104 has no AC
- [MEDIUM] Vague term: "should" used 5 times without specifics
...Step 5: Record Learning
After pipeline execution, record the outcome for learning:
// Store validation pattern
memory store --namespace validation-pipeline --key "req-validation-{timestamp}" --value "{score, findings_count, halted}"Parameters
| Parameter | Type | Default | Description |
|---|---|---|---|
pipeline | string | requirements | Pipeline type to run |
file | string | required | Path to document to validate |
--steps | string | all | Comma-separated step IDs to run (e.g., format-check,completeness-check) |
--continue-on-failure | boolean | false | Skip blocking gates |
--json | boolean | false | Output as JSON instead of markdown |
--metadata | object | {} | Additional context for steps |
Integration Points
- qe-requirements-validator agent — delegates structured validation to this pipeline
- qe-quality-gate agent — uses pipeline for gate evaluation
- YAML Pipelines — can invoke validation steps as workflow actions
- MCP — accessible via
pipeline_validatetool
Output Schema
The pipeline produces a PipelineResult object (see schemas/output.json):
{
pipelineId: string;
pipelineName: string;
overall: 'pass' | 'fail' | 'warn';
score: number; // 0-100 weighted average
steps: StepResult[]; // per-step details
blockers: Finding[]; // blocking findings
halted: boolean;
haltedAt?: string; // step ID where halted
totalDuration: number;
timestamp: string;
}Error Handling
- Step throws exception — captured as a FAIL with critical finding, pipeline continues or halts per severity
- File not found — report error, do not run pipeline
- Empty document — format-check step will catch this as a blocking failure
# =============================================================================
# AQE Skill Evaluation Test Suite: Validation Pipeline v1.0.0
# =============================================================================
#
# Comprehensive evaluation suite for the validation-pipeline skill.
# Tests structured step-by-step validation with gate enforcement,
# per-step scoring, and report generation.
#
# Schema: .claude/skills/.validation/schemas/skill-eval.schema.json
# Validator: .claude/skills/validation-pipeline/scripts/validate-config.json
#
# Coverage:
# - Full requirements pipeline execution (13 steps)
# - Blocking gate enforcement (halt on failure)
# - Continue-on-failure mode
# - Step filtering (--steps)
# - Per-step scoring and weighted rollup
# - Report generation (markdown and JSON)
#
# =============================================================================
skill: validation-pipeline
version: 1.0.0
description: >
Evaluation suite for the validation-pipeline skill. Tests full pipeline
execution, gate enforcement, step filtering, scoring, and reporting
across the 13-step requirements validation pipeline.
# =============================================================================
# Multi-Model Configuration
# =============================================================================
models_to_test:
- claude-sonnet-4-6 # Primary (high accuracy expected)
- claude-haiku-4-5 # Fast model (minimum quality floor)
# =============================================================================
# MCP Integration Configuration
# =============================================================================
mcp_integration:
enabled: true
namespace: skill-validation
query_patterns: true
track_outcomes: true
store_patterns: true
share_learning: true
update_quality_gate: true
target_agents:
- qe-requirements-validator
- qe-quality-gate
# =============================================================================
# Learning Configuration
# =============================================================================
learning:
store_success_patterns: true
store_failure_patterns: true
pattern_ttl_days: 90
min_confidence_to_store: 0.7
cross_model_comparison: true
# =============================================================================
# Result Format
# =============================================================================
result_format:
json_output: true
markdown_report: true
include_raw_output: false
include_timing: true
include_token_usage: true
# =============================================================================
# Setup
# =============================================================================
setup:
required_tools: []
environment_variables: {}
fixtures: []
# =============================================================================
# TEST CASES
# =============================================================================
test_cases:
# ---------------------------------------------------------------------------
# CATEGORY: Full Pipeline Execution
# ---------------------------------------------------------------------------
- id: tc001_full_requirements_pipeline
description: "Execute all 13 requirements validation steps on a well-formed document"
category: pipeline_execution
priority: critical
input:
prompt: |
Run the validation pipeline on this requirements document:
# User Management Requirements
## Overview
This document specifies the requirements for the user management module.
## Requirements
### REQ-001: User Registration
As a visitor, I want to register an account so I can access the platform.
**Acceptance Criteria:**
- User can register with email and password (8+ chars, 1 uppercase, 1 number)
- Duplicate email rejected with clear error message
- Verification email sent within 30 seconds
- Registration completes in under 2 seconds
### REQ-002: User Login
As a registered user, I want to log in so I can access my account.
**Acceptance Criteria:**
- User can log in with email and password
- Invalid credentials show generic error (no email enumeration)
- Account locks after 5 failed attempts for 15 minutes
- Session expires after 30 minutes of inactivity
## Scope
In scope: registration, login, password reset.
Out of scope: social login, SSO.
context:
pipeline: "requirements"
expected_output:
must_contain:
- "pipeline"
- "validation"
- "score"
- "step"
- "finding"
must_not_contain:
- "error"
- "unable"
severity_classification: critical
validation:
schema_check: true
keyword_match_threshold: 0.8
reasoning_quality_min: 0.75
- id: tc002_blocking_gate_enforcement
description: "Pipeline halts at blocking step failure"
category: gate_enforcement
priority: critical
input:
prompt: |
Run the validation pipeline on this minimal document:
just some text without any structure
The pipeline should halt at the format-check step (blocking)
because there are no headings, required sections, or structure.
context:
pipeline: "requirements"
expected_output:
must_contain:
- "halt"
- "block"
- "format"
- "fail"
severity_classification: critical
finding_count:
min: 1
validation:
schema_check: true
keyword_match_threshold: 0.7
- id: tc003_continue_on_failure
description: "Pipeline continues past blocking failures with --continue-on-failure"
category: gate_enforcement
priority: high
input:
prompt: |
Run the validation pipeline with --continue-on-failure on this document:
just some text without any structure
Even though format-check will fail (blocking), the pipeline should
continue executing remaining steps and report all findings.
context:
pipeline: "requirements"
continue_on_failure: true
expected_output:
must_contain:
- "continue"
- "format"
- "step"
- "finding"
finding_count:
min: 2
validation:
schema_check: true
keyword_match_threshold: 0.7
# ---------------------------------------------------------------------------
# CATEGORY: Step Filtering
# ---------------------------------------------------------------------------
- id: tc004_step_filtering
description: "Run only specific steps from the pipeline"
category: step_filtering
priority: high
input:
prompt: |
Run only the format-check and vague-term-detection steps on:
# Requirements Document
## Overview
The system should handle various types of user input properly.
## Requirements
Users might want to do several things with the system.
## Scope
The system should support etc.
context:
pipeline: "requirements"
steps: ["format-check", "vague-term-detection"]
expected_output:
must_contain:
- "format"
- "vague"
- "should"
- "step"
finding_count:
min: 1
validation:
schema_check: true
keyword_match_threshold: 0.75
# ---------------------------------------------------------------------------
# CATEGORY: Scoring and Reporting
# ---------------------------------------------------------------------------
- id: tc005_weighted_score_calculation
description: "Overall score uses category-weighted averages"
category: scoring
priority: high
input:
prompt: |
Run the full requirements pipeline and verify the scoring breakdown:
# API Gateway Requirements
## Overview
Requirements for the API gateway service.
## Requirements
### REQ-001: Rate Limiting
Limit API requests to 1000/minute per client.
**Acceptance Criteria:**
- Requests beyond limit return 429 status
- Rate limit headers included in all responses
- Configurable per-endpoint limits
## Scope
Rate limiting, authentication proxy, request routing.
Show the per-step scores and how the overall weighted score is calculated.
context:
pipeline: "requirements"
expected_output:
must_contain:
- "score"
- "weight"
- "step"
- "overall"
validation:
schema_check: true
keyword_match_threshold: 0.75
- id: tc006_markdown_report_format
description: "Pipeline produces well-formatted markdown report"
category: reporting
priority: high
input:
prompt: |
Run the requirements pipeline and output a markdown report:
# Payment Processing
## Overview
Handle payment transactions securely.
## Requirements
### REQ-001: Card Payment
Process credit/debit card payments via Stripe.
**Acceptance Criteria:**
- Support Visa, Mastercard, Amex
- PCI DSS compliant tokenization
- Transaction completes in under 5 seconds
## Scope
Card payments only. Wire transfers out of scope.
context:
pipeline: "requirements"
format: "markdown"
expected_output:
must_contain:
- "Validation Report"
- "Step Results"
- "Score"
- "Duration"
- "Finding"
validation:
schema_check: true
keyword_match_threshold: 0.8
# ---------------------------------------------------------------------------
# CATEGORY: Individual Step Validation
# ---------------------------------------------------------------------------
- id: tc007_invest_criteria_check
description: "INVEST criteria step evaluates requirement quality"
category: individual_steps
priority: high
input:
prompt: |
Run only the invest-criteria step on:
# Requirements
## Overview
System requirements for user management.
## Requirements
### REQ-001: User CRUD
The system shall provide full CRUD operations for user entities including
create, read, update, and delete with proper authorization checks,
audit logging, soft delete support, batch operations, import/export,
admin override, and integration with 5 external systems.
## Scope
User management module.
Check: is REQ-001 Independent, Negotiable, Valuable, Estimable, Small, Testable?
context:
pipeline: "requirements"
steps: ["invest-criteria"]
expected_output:
must_contain:
- "INVEST"
- "small"
- "testable"
- "independent"
finding_count:
min: 1
validation:
schema_check: true
keyword_match_threshold: 0.7
- id: tc008_vague_term_detection
description: "Detects vague and ambiguous terms"
category: individual_steps
priority: high
input:
prompt: |
Run only the vague-term-detection step on:
# Requirements
## Overview
The system should handle various scenarios properly.
## Requirements
The platform might need to support several user types etc.
Performance should be adequate for most use cases.
The UI should be user-friendly and intuitive.
## Scope
Various features and improvements.
context:
pipeline: "requirements"
steps: ["vague-term-detection"]
expected_output:
must_contain:
- "vague"
- "should"
- "various"
- "etc"
finding_count:
min: 3
validation:
schema_check: true
keyword_match_threshold: 0.75
# ---------------------------------------------------------------------------
# CATEGORY: Negative / Edge Cases
# ---------------------------------------------------------------------------
- id: tc009_empty_document
description: "Pipeline handles empty document gracefully"
category: negative
priority: high
input:
prompt: |
Run the requirements pipeline on an empty document (no content).
context:
pipeline: "requirements"
content: ""
expected_output:
must_contain:
- "fail"
- "empty"
- "format"
severity_classification: critical
finding_count:
min: 1
validation:
schema_check: true
allow_partial: true
- id: tc010_high_quality_document
description: "High-quality document scores well across all steps"
category: positive
priority: high
input:
prompt: |
Run the full pipeline on this well-structured requirements document:
# Authentication Service Requirements v2.1
## Overview
This document specifies authentication requirements for the platform.
All requirements have been reviewed by the security team and product owner.
## Requirements
### REQ-001: OAuth2 Login
As a user, I want to authenticate via OAuth2 providers so I can use
existing credentials without creating a new password.
**Acceptance Criteria:**
- Given a user clicks "Sign in with Google", when they authorize,
then they are redirected to the dashboard within 3 seconds
- Given an invalid OAuth token, when login is attempted,
then the system returns a 401 with descriptive error
- Given a new OAuth user, when they first authenticate,
then a local account is created automatically
**Tests:** TC-001, TC-002, TC-003
**Dependencies:** REQ-005 (Session Management)
### REQ-002: Password Requirements
As a security administrator, I want password complexity rules enforced
so that user accounts are protected against brute force attacks.
**Acceptance Criteria:**
- Minimum 12 characters, 1 uppercase, 1 lowercase, 1 number, 1 symbol
- Password strength meter shows real-time feedback
- Common passwords (top 10,000) are rejected with suggestion
- Password history prevents reuse of last 5 passwords
**Tests:** TC-010, TC-011, TC-012, TC-013
**Dependencies:** None
## Scope
In scope: OAuth2, password management, MFA.
Out of scope: Biometric authentication, hardware keys.
context:
pipeline: "requirements"
expected_output:
must_contain:
- "pass"
- "score"
- "step"
- "validation"
severity_classification: high
validation:
schema_check: true
keyword_match_threshold: 0.8
reasoning_quality_min: 0.8
# =============================================================================
# SUCCESS CRITERIA
# =============================================================================
success_criteria:
pass_rate: 0.8
critical_pass_rate: 1.0
avg_reasoning_quality: 0.75
max_execution_time_ms: 300000
cross_model_variance: 0.15
# =============================================================================
# METADATA
# =============================================================================
metadata:
author: "qe-requirements-validator"
created: "2026-03-12"
last_updated: "2026-03-12"
coverage_target: >
Full 13-step requirements pipeline execution, blocking gate enforcement,
continue-on-failure mode, step filtering, weighted score calculation,
markdown report generation, INVEST criteria validation, vague term detection,
empty document handling, and high-quality document scoring.
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://agentic-qe.dev/schemas/validation-pipeline-output.json",
"title": "AQE Validation Pipeline Skill Output Schema",
"description": "Schema for validation pipeline skill output. Includes step results, scores, findings, and blockers.",
"type": "object",
"required": ["skillName", "version", "timestamp", "status", "trustTier", "output"],
"properties": {
"skillName": {
"type": "string",
"const": "validation-pipeline",
"description": "Must be 'validation-pipeline'"
},
"version": {
"type": "string",
"pattern": "^\\d+\\.\\d+\\.\\d+(-[a-zA-Z0-9]+)?$"
},
"timestamp": {
"type": "string",
"format": "date-time"
},
"status": {
"type": "string",
"enum": ["success", "partial", "failed", "skipped"]
},
"trustTier": {
"type": "integer",
"const": 3,
"description": "Trust tier 3 indicates full validation with eval suite"
},
"output": {
"type": "object",
"required": ["pipelineId", "pipelineName", "overall", "score", "steps"],
"properties": {
"pipelineId": {
"type": "string",
"description": "Pipeline identifier"
},
"pipelineName": {
"type": "string",
"description": "Human-readable pipeline name"
},
"overall": {
"type": "string",
"enum": ["pass", "fail", "warn"],
"description": "Overall pipeline verdict"
},
"score": {
"type": "integer",
"minimum": 0,
"maximum": 100,
"description": "Weighted average score across all steps"
},
"steps": {
"type": "array",
"items": { "$ref": "#/$defs/stepResult" },
"minItems": 1,
"description": "Per-step results"
},
"blockers": {
"type": "array",
"items": { "$ref": "#/$defs/finding" },
"description": "Findings from blocking steps that failed"
},
"halted": {
"type": "boolean",
"description": "Whether pipeline halted early"
},
"haltedAt": {
"type": "string",
"description": "Step ID where pipeline halted"
},
"totalDuration": {
"type": "integer",
"minimum": 0,
"description": "Total execution time in ms"
},
"summary": {
"type": "string",
"minLength": 20,
"maxLength": 2000,
"description": "Human-readable summary"
}
}
},
"metadata": { "$ref": "#/$defs/metadata" },
"validation": { "$ref": "#/$defs/validationResult" },
"learning": { "$ref": "#/$defs/learningData" }
},
"$defs": {
"stepResult": {
"type": "object",
"required": ["stepId", "stepName", "status", "score", "findings", "evidence", "duration"],
"properties": {
"stepId": {
"type": "string",
"description": "Step identifier"
},
"stepName": {
"type": "string",
"description": "Human-readable step name"
},
"status": {
"type": "string",
"enum": ["pass", "fail", "warn", "skip"],
"description": "Step verdict"
},
"score": {
"type": "integer",
"minimum": 0,
"maximum": 100,
"description": "Step score"
},
"findings": {
"type": "array",
"items": { "$ref": "#/$defs/finding" }
},
"evidence": {
"type": "array",
"items": { "type": "string" },
"description": "What was checked"
},
"duration": {
"type": "integer",
"minimum": 0,
"description": "Step duration in ms"
}
}
},
"finding": {
"type": "object",
"required": ["id", "stepId", "severity", "title", "description"],
"properties": {
"id": {
"type": "string"
},
"stepId": {
"type": "string"
},
"severity": {
"type": "string",
"enum": ["critical", "high", "medium", "low", "info"]
},
"title": {
"type": "string",
"minLength": 5,
"maxLength": 200
},
"description": {
"type": "string",
"maxLength": 2000
},
"location": {
"type": "string",
"description": "File path or document section"
},
"suggestion": {
"type": "string",
"description": "Suggested fix"
}
}
},
"metadata": {
"type": "object",
"properties": {
"executionTimeMs": { "type": "integer", "minimum": 0 },
"toolsUsed": { "type": "array", "items": { "type": "string" } },
"agentId": { "type": "string" },
"pipelineType": { "type": "string" },
"targetFile": { "type": "string" },
"stepsExecuted": { "type": "integer", "minimum": 0 },
"stepsSkipped": { "type": "integer", "minimum": 0 }
}
},
"validationResult": {
"type": "object",
"properties": {
"schemaValid": { "type": "boolean" },
"contentValid": { "type": "boolean" },
"confidence": { "type": "number", "minimum": 0, "maximum": 1 },
"warnings": { "type": "array", "items": { "type": "string" }, "maxItems": 20 },
"errors": { "type": "array", "items": { "type": "string" }, "maxItems": 20 }
}
},
"learningData": {
"type": "object",
"properties": {
"patternsDetected": { "type": "array", "items": { "type": "string" }, "maxItems": 20 },
"reward": { "type": "number", "minimum": 0, "maximum": 1 }
}
}
}
}
{
"skillName": "validation-pipeline",
"skillVersion": "1.0.0",
"requiredTools": [],
"optionalTools": ["jq"],
"schemaPath": "schemas/output.json",
"requiredFields": [
"skillName",
"status",
"output",
"output.pipelineId",
"output.pipelineName",
"output.overall",
"output.score",
"output.steps"
],
"requiredNonEmptyFields": [
"output.pipelineName",
"output.steps"
],
"mustContainTerms": [
"pipeline",
"validation"
],
"mustNotContainTerms": [
"TODO",
"placeholder",
"FIXME"
],
"enumValidations": {
".status": ["success", "partial", "failed", "skipped"],
".output.overall": ["pass", "fail", "warn"]
}
}
Related skills
Setup Matt Pocock SkillsScaffold the per-repo configuration that Matt Pocock’s engineering agent skills rely on so they understand the issue tracker, triage labels, and domain documentation la495k196k
Lark AppsConnect Claude, Cursor or custom agents directly to Lark (Feishu) for messaging, document automation, approval workflows and enterprise data access.402k
CavemanSlash token usage by roughly 75% while keeping every technical detail intact when working with Claude Code, Cursor or similar agents.391k94.7k
Lark Skill MakerQuickly turn any Lark/Feishu OpenAPI call or multi-step workflow into a reusable agent skill with its own SKILL.md.386k16k
Running Claude Code Via Litellm CopilotRun Claude Code at a fraction of the cost by routing requests through LiteLLM to the GitHub Copilot Chat API.270k72
Codex PetGenerate a complete Codex Pet spritesheet and metadata from one reference image without needing an OpenAI key or Codex Pro.246k10FAQ
What does validation-pipeline do?
validation-pipeline is a Claude Code skill for ai & agent building.
When should I use validation-pipeline?
When you need to helps with ai & agent building tasks., or when validation-pipeline is a claude code skill for ai & agent building.
What are the main capabilities?
validation-pipeline; AI & Agent Building; AI-coding skill.
AI & Agent Buildingagents