
Contract Testing
- 106 installs
- 433 repo stars
- Updated August 4, 2026
- proffesor-for-testing/agentic-qe
contract-testing is a Claude Code skill for testing & qa.
About
contract-testing is a Claude Code skill for testing & qa. It helps solo builders move faster with AI-assisted development.
- contract-testing
- Testing & QA
- AI-coding skill
Contract Testing by the numbers
- 106 all-time installs (skills.sh)
- +4 installs in the week ending Aug 4, 2026 (Skillselion tracking)
- Ranked #978 of 2,153 Testing & QA skills by installs in the Skillselion catalog
- Data as of Aug 5, 2026 (Skillselion catalog sync)
npx skills add https://github.com/proffesor-for-testing/agentic-qe --skill contract-testingAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 106 |
|---|---|
| repo stars | ★ 433 |
| Last updated | August 4, 2026 |
| Repository | proffesor-for-testing/agentic-qe ↗ |
How do I helps with testing & qa tasks.?
Helps with testing & qa tasks.
Who is it for?
Best when you're working on testing & qa and need structured help with contract testing.
Skip if: Teams with no testing & qa needs, or anyone wanting a generic chat assistant without this specific workflow.
When should I use this skill?
When you need to helps with testing & qa tasks., or when contract-testing is a claude code skill for testing & qa.
What you get
Structured output aligned to contract-testing: contract-testing, Testing & QA.
Files
Contract Testing
<default_to_action> When testing API contracts or microservices: 1. DEFINE consumer expectations (what consumers actually need) 2. VERIFY provider fulfills contracts (Pact verification) 3. DETECT breaking changes before deployment (CI/CD integration) 4. VERSION APIs semantically (breaking = major bump) 5. MAINTAIN backward compatibility for supported versions
Quick Contract Testing Steps:
- Consumer: Define expected request/response pairs
- Provider: Verify against all consumer contracts
- CI/CD: Block deploys that break contracts
- Versioning: Document supported versions and deprecation
Critical Success Factors:
- Consumers own the contract (they define what they need)
- Provider must pass all consumer contracts before deploy
- Breaking changes require coordination, not surprise
</default_to_action>
Quick Reference Card
When to Use
- Microservices communication
- Third-party API integrations
- Distributed team coordination
- Preventing breaking changes
Consumer-Driven Contract Flow
Consumer → Defines Expectations → Contract
↓
Provider → Verifies Contract → Pass/Fail
↓
CI/CD → Blocks Breaking ChangesBreaking vs Non-Breaking Changes
| Change Type | Breaking? | Semver |
|---|---|---|
| Remove field | ✅ Yes | Major |
| Rename field | ✅ Yes | Major |
| Change type | ✅ Yes | Major |
| Add optional field | ❌ No | Minor |
| Add new endpoint | ❌ No | Minor |
| Bug fix | ❌ No | Patch |
Tools
| Tool | Best For |
|---|---|
| Pact | Consumer-driven contracts |
| OpenAPI/Swagger | API-first design |
| JSON Schema | Schema validation |
| GraphQL | Schema-first contracts |
---
Consumer Contract (Pact)
// Consumer defines what it needs
const { Pact } = require('@pact-foundation/pact');
describe('Order API Consumer', () => {
const provider = new Pact({
consumer: 'CheckoutUI',
provider: 'OrderService'
});
beforeAll(() => provider.setup());
afterAll(() => provider.finalize());
it('creates an order', async () => {
await provider.addInteraction({
state: 'products exist',
uponReceiving: 'a create order request',
withRequest: {
method: 'POST',
path: '/orders',
body: { productId: 'abc', quantity: 2 }
},
willRespondWith: {
status: 201,
body: {
orderId: like('order-123'), // Any string matching pattern
total: like(19.99) // Any number
}
}
});
const response = await orderClient.create({ productId: 'abc', quantity: 2 });
expect(response.orderId).toBeDefined();
});
});---
Provider Verification
// Provider verifies it fulfills all consumer contracts
const { Verifier } = require('@pact-foundation/pact');
describe('Order Service Provider', () => {
it('fulfills all consumer contracts', async () => {
await new Verifier({
provider: 'OrderService',
providerBaseUrl: 'http://localhost:3000',
pactUrls: ['./pacts/checkoutui-orderservice.json'],
stateHandlers: {
'products exist': async () => {
await db.products.create({ id: 'abc', price: 9.99 });
}
}
}).verifyProvider();
});
});---
Breaking Change Detection
// Agent detects breaking changes
await Task("Contract Validation", {
currentContract: 'openapi-v2.yaml',
previousContract: 'openapi-v1.yaml',
detectBreaking: true,
calculateSemver: true,
generateMigrationGuide: true
}, "qe-api-contract-validator");
// Output:
// Breaking changes found: 2
// - Removed field: order.discount
// - Type change: order.total (number → string)
// Recommended version: 3.0.0 (major bump)---
CI/CD Integration
name: Contract Tests
on: [push]
jobs:
consumer-tests:
steps:
- run: npm run test:contract
- name: Publish Pacts
run: npx pact-broker publish ./pacts --broker-base-url $PACT_BROKER
provider-verification:
needs: consumer-tests
steps:
- name: Verify Provider
run: npm run verify:contracts
- name: Can I Deploy?
run: npx pact-broker can-i-deploy --pacticipant OrderService --version $VERSION---
Agent Coordination Hints
Memory Namespace
aqe/contract-testing/
├── contracts/* - Current contracts
├── breaking-changes/* - Detected breaking changes
├── versioning/* - Version compatibility matrix
└── verification-results/* - Provider verification historyFleet Coordination
const contractFleet = await FleetManager.coordinate({
strategy: 'contract-testing',
agents: [
'qe-api-contract-validator', // Validation, breaking detection
'qe-test-generator', // Generate contract tests
'qe-security-scanner' // API security
],
topology: 'sequential'
});---
Agent CLI & Advanced Patterns
For v3 agent-specific commands (aqe contract ...), GraphQL contracts, event contracts, and Pact Broker integration, see references/agent-commands.md.
Related Skills
- api-testing-patterns - API testing strategies
- shift-left-testing - Early contract validation
- cicd-pipeline-qe-orchestrator - Pipeline integration
---
Remember
Consumers own the contract. They define what they need; providers must fulfill it. Breaking changes require major version bumps and coordination. CI/CD blocks deploys that break contracts. Use Pact for consumer-driven, OpenAPI for API-first.
With Agents: Agents validate contracts, detect breaking changes with semver recommendations, and generate migration guides. Use agents to maintain contract compliance at scale.
Gotchas
- Pact broker URL must be configured before running — agent will generate tests that silently skip verification without it
- Consumer tests pass locally but fail in CI when provider states aren't set up — always verify both sides
- Adding a required field to a response is a BREAKING change even though provider tests pass — consumer didn't expect it
- Agent may generate contracts from API docs instead of actual consumer usage — contracts must reflect real consumer needs
- GraphQL contract testing requires schema stitching awareness — fragments may reference types from other services
{
"$schema": "./config-schema.json",
"_description": "Contract Testing configuration. Auto-created on first run. Edit to customize.",
"broker_url": null,
"consumer_name": null,
"provider_name": null,
"options": {
"publishVerificationResults": true,
"enablePending": true,
"includeWipPactsSince": null
},
"_setupPrompt": "If broker_url is null, ask: 'What is your Pact Broker URL? (or \"local\" for file-based)'. If consumer_name is null, ask: 'What is the consumer service name?'. If provider_name is null, ask: 'What is the provider service name?'"
}
# =============================================================================
# AQE Contract Testing Skill Evaluation Test Suite v1.0.0
# Per ADR-056 - Trust Tier 3 Validation
# =============================================================================
#
# This evaluation suite validates the contract-testing skill behavior:
# - Consumer-driven contract testing (Pact)
# - Provider verification
# - Breaking change detection
# - Semantic versioning recommendations
# - Spring Cloud Contract support
# - canIDeploy decision logic
# - Mock generation from contracts
#
# Schema: .claude/skills/.validation/schemas/skill-eval.schema.json
# Runner: scripts/run-skill-eval.ts
#
# =============================================================================
skill: contract-testing
version: 1.0.0
description: >
Comprehensive evaluation suite for the contract-testing skill.
Tests consumer-driven contracts, provider verification, breaking change
detection, versioning recommendations, and CI/CD integration patterns
across multiple models.
# =============================================================================
# Multi-Model Configuration
# =============================================================================
models_to_test:
- claude-sonnet-4-6 # Primary (high accuracy expected)
- claude-haiku-4-5 # Fast model (minimum quality floor)
# =============================================================================
# MCP Integration Configuration
# =============================================================================
mcp_integration:
enabled: true
namespace: skill-validation
query_patterns: true
track_outcomes: true
store_patterns: true
share_learning: true
update_quality_gate: true
target_agents:
- qe-learning-coordinator
- qe-queen-coordinator
- qe-api-contract-validator
# =============================================================================
# ReasoningBank Learning Configuration
# =============================================================================
learning:
store_success_patterns: true
store_failure_patterns: true
pattern_ttl_days: 90
min_confidence_to_store: 0.7
cross_model_comparison: true
# =============================================================================
# Result Format Configuration
# =============================================================================
result_format:
json_output: true
markdown_report: true
include_raw_output: false
include_timing: true
include_token_usage: true
# =============================================================================
# Environment Setup
# =============================================================================
setup:
required_tools:
- jq
environment_variables:
AQE_VALIDATION_MODE: "eval"
fixtures:
- name: sample_pact_file
path: fixtures/pact-sample.json
content: |
{
"consumer": { "name": "CheckoutUI" },
"provider": { "name": "OrderService" },
"interactions": [
{
"description": "a request to create an order",
"providerState": "products exist",
"request": {
"method": "POST",
"path": "/orders",
"headers": { "Content-Type": "application/json" },
"body": { "productId": "abc-123", "quantity": 2 }
},
"response": {
"status": 201,
"headers": { "Content-Type": "application/json" },
"body": {
"orderId": "order-456",
"total": 19.99
}
}
}
],
"metadata": {
"pactSpecification": { "version": "4.0" }
}
}
- name: sample_openapi_v1
path: fixtures/openapi-v1.yaml
content: |
openapi: "3.0.3"
info:
title: Order API
version: "1.0.0"
paths:
/orders:
post:
operationId: createOrder
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/CreateOrderRequest'
responses:
"201":
description: Order created
content:
application/json:
schema:
$ref: '#/components/schemas/Order'
components:
schemas:
CreateOrderRequest:
type: object
required: [productId, quantity]
properties:
productId:
type: string
quantity:
type: integer
Order:
type: object
properties:
orderId:
type: string
total:
type: number
discount:
type: number
- name: sample_openapi_v2
path: fixtures/openapi-v2.yaml
content: |
openapi: "3.0.3"
info:
title: Order API
version: "2.0.0"
paths:
/orders:
post:
operationId: createOrder
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/CreateOrderRequest'
responses:
"201":
description: Order created
content:
application/json:
schema:
$ref: '#/components/schemas/Order'
components:
schemas:
CreateOrderRequest:
type: object
required: [productId, quantity, customerId]
properties:
productId:
type: string
quantity:
type: integer
customerId:
type: string
Order:
type: object
properties:
orderId:
type: string
total:
type: string
# =============================================================================
# Test Cases
# =============================================================================
test_cases:
# -------------------------------------------------------------------------
# Consumer Contract Generation Tests
# -------------------------------------------------------------------------
- id: tc001_consumer_contract_generation
description: "Skill generates consumer contract from API interaction"
category: consumer
priority: critical
input:
prompt: |
Generate a Pact consumer contract for the following API interaction:
- Consumer: web-app
- Provider: user-service
- Interaction: GET /users/{id} returns user details
- Expected response: { "id": "123", "name": "John", "email": "john@example.com" }
context:
contractType: consumer-driven
framework: pact
expected_output:
must_contain:
- "consumer"
- "provider"
- "web-app"
- "user-service"
- "GET"
- "users"
must_not_contain:
- "error"
- "unable"
- "TODO"
validation:
schema_check: true
keyword_match_threshold: 0.8
reasoning_quality_min: 0.7
- id: tc002_pact_matchers
description: "Skill correctly uses Pact matchers for flexible contracts"
category: consumer
priority: high
input:
prompt: |
Create a Pact contract with flexible matching for:
- Any string for orderId (should match any string pattern)
- Any positive number for total
- Array of items where each has id and name
context:
framework: pact
pactVersion: "4.0"
expected_output:
must_contain:
- "matching"
- "type"
must_match_regex:
- "(?i)(like|regex|matcher|type.*match)"
validation:
schema_check: true
keyword_match_threshold: 0.7
# -------------------------------------------------------------------------
# Provider Verification Tests
# -------------------------------------------------------------------------
- id: tc003_provider_verification
description: "Skill verifies provider against consumer contracts"
category: provider
priority: critical
input:
prompt: |
Verify the user-service provider against all consumer contracts:
- Consumers: web-app (v1.2.0), mobile-app (v2.0.1), admin-portal (v1.0.0)
- Provider version: 3.1.0
- Pact Broker URL: https://pact-broker.example.com
context:
contractType: consumer-driven
framework: pact
expected_output:
must_contain:
- "verification"
- "provider"
- "consumer"
- "user-service"
must_match_regex:
- "(?i)(pass|fail|verify)"
validation:
schema_check: true
keyword_match_threshold: 0.8
reasoning_quality_min: 0.8
- id: tc004_provider_states
description: "Skill handles provider states correctly"
category: provider
priority: high
input:
prompt: |
Create provider state handlers for the following contract:
- State 1: "user 123 exists" - need to seed user with id 123
- State 2: "no users exist" - need to clear database
- State 3: "user has orders" - need user and associated orders
context:
framework: pact
language: javascript
expected_output:
must_contain:
- "state"
- "handler"
- "user"
must_match_regex:
- "(?i)(setup|teardown|before|seed|clear)"
validation:
schema_check: true
keyword_match_threshold: 0.7
# -------------------------------------------------------------------------
# Breaking Change Detection Tests
# -------------------------------------------------------------------------
- id: tc005_breaking_change_removed_field
description: "Skill detects breaking change when field is removed"
category: breaking_change
priority: critical
input:
prompt: |
Compare these two API versions and identify breaking changes:
V1 Response:
{
"orderId": "123",
"total": 99.99,
"discount": 10.00,
"items": []
}
V2 Response:
{
"orderId": "123",
"total": "99.99",
"items": []
}
context:
detectBreaking: true
expected_output:
must_contain:
- "breaking"
- "removed"
- "discount"
- "type"
must_match_regex:
- "(?i)(breaking.*change|removed.*field|type.*change)"
validation:
schema_check: true
keyword_match_threshold: 0.9
grading_rubric:
completeness: 0.4
accuracy: 0.4
actionability: 0.2
- id: tc006_breaking_change_type_change
description: "Skill detects type change as breaking"
category: breaking_change
priority: critical
input:
prompt: |
Is changing a field from number to string a breaking change?
Example: "total: 99.99" changed to "total: '99.99'"
What consumers might be affected?
context:
detectBreaking: true
expected_output:
must_contain:
- "breaking"
- "type"
- "number"
- "string"
must_match_regex:
- "(?i)(breaking|incompatible|consumer.*impact)"
validation:
schema_check: true
keyword_match_threshold: 0.8
- id: tc007_non_breaking_change
description: "Skill correctly identifies non-breaking changes"
category: breaking_change
priority: high
input:
prompt: |
Are these changes breaking or non-breaking?
1. Adding a new optional field "createdAt"
2. Adding a new endpoint GET /orders/history
3. Adding a new enum value "PENDING_APPROVAL" to status field
context:
detectBreaking: true
expected_output:
must_contain:
- "non-breaking"
- "optional"
- "add"
must_not_contain:
- "breaking change"
- "incompatible"
validation:
schema_check: true
keyword_match_threshold: 0.7
# -------------------------------------------------------------------------
# Semantic Versioning Tests
# -------------------------------------------------------------------------
- id: tc008_semver_major_bump
description: "Skill recommends major version bump for breaking changes"
category: versioning
priority: high
input:
prompt: |
Current API version: 1.5.3
Changes detected:
- Removed field "legacyId" from User response
- Changed "createdAt" from string to timestamp
What version should we release?
context:
currentVersion: "1.5.3"
calculateSemver: true
expected_output:
must_contain:
- "2.0.0"
- "major"
- "breaking"
must_not_contain:
- "1.5.4"
- "1.6.0"
- "patch"
validation:
schema_check: true
keyword_match_threshold: 0.9
- id: tc009_semver_minor_bump
description: "Skill recommends minor version bump for new features"
category: versioning
priority: high
input:
prompt: |
Current API version: 2.3.1
Changes:
- Added new endpoint GET /orders/summary
- Added optional field "metadata" to Order response
Recommend the next version.
context:
currentVersion: "2.3.1"
calculateSemver: true
expected_output:
must_contain:
- "2.4.0"
- "minor"
must_not_contain:
- "3.0.0"
- "major"
- "breaking"
validation:
schema_check: true
keyword_match_threshold: 0.8
# -------------------------------------------------------------------------
# canIDeploy Tests
# -------------------------------------------------------------------------
- id: tc010_can_i_deploy_yes
description: "Skill correctly determines deployment is safe"
category: deployment
priority: critical
input:
prompt: |
Provider: order-service v2.1.0
Consumer contracts verified:
- web-app v1.5.0: PASSED (12/12 interactions)
- mobile-app v2.0.1: PASSED (8/8 interactions)
- admin-portal v1.0.0: PASSED (5/5 interactions)
Can I deploy order-service to production?
context:
canIDeploy: true
expected_output:
must_contain:
- "deploy"
- "pass"
- "safe"
must_match_regex:
- "(?i)(can.*deploy|safe.*deploy|yes)"
validation:
schema_check: true
keyword_match_threshold: 0.8
- id: tc011_can_i_deploy_no
description: "Skill correctly blocks deployment when contracts fail"
category: deployment
priority: critical
input:
prompt: |
Provider: payment-service v3.0.0
Consumer contracts verified:
- checkout-ui v2.1.0: PASSED (10/10)
- mobile-app v1.9.0: FAILED (7/8 - missing field)
- reporting-service v1.0.0: PASSED (3/3)
Can I deploy payment-service to production?
context:
canIDeploy: true
expected_output:
must_contain:
- "cannot"
- "deploy"
- "fail"
- "mobile-app"
must_match_regex:
- "(?i)(cannot.*deploy|do.*not.*deploy|block|fail)"
validation:
schema_check: true
keyword_match_threshold: 0.9
# -------------------------------------------------------------------------
# Spring Cloud Contract Tests
# -------------------------------------------------------------------------
- id: tc012_spring_cloud_contract
description: "Skill generates Spring Cloud Contract DSL"
category: spring_cloud
priority: medium
input:
prompt: |
Generate a Spring Cloud Contract for:
- Request: POST /api/users with body { "name": "John", "email": "john@test.com" }
- Response: 201 with body { "id": "generated-uuid", "name": "John" }
Use Groovy DSL format.
context:
framework: spring-cloud-contract
language: groovy
expected_output:
must_contain:
- "Contract"
- "request"
- "response"
- "POST"
- "201"
must_match_regex:
- "(?i)(contract|dsl|groovy)"
validation:
schema_check: true
keyword_match_threshold: 0.7
# -------------------------------------------------------------------------
# Mock Generation Tests
# -------------------------------------------------------------------------
- id: tc013_mock_generation
description: "Skill generates mock from contract for development"
category: mock
priority: high
input:
prompt: |
Generate a WireMock stub from this Pact interaction:
Request: GET /api/users/123
Response: 200 with { "id": "123", "name": "John", "status": "active" }
Include matching rules for flexible stub.
context:
mockType: wiremock
expected_output:
must_contain:
- "stub"
- "request"
- "response"
- "200"
must_match_regex:
- "(?i)(wiremock|stub|mapping|mock)"
validation:
schema_check: true
keyword_match_threshold: 0.7
# -------------------------------------------------------------------------
# Pact Broker Integration Tests
# -------------------------------------------------------------------------
- id: tc014_pact_broker_publish
description: "Skill explains Pact Broker publishing workflow"
category: broker
priority: medium
input:
prompt: |
How do I publish consumer Pact files to a Pact Broker?
What tags and versions should I use?
CI/CD integration best practices?
context:
framework: pact
environment: ci
expected_output:
must_contain:
- "publish"
- "broker"
- "version"
- "tag"
must_match_regex:
- "(?i)(pact.*broker|publish|ci.*cd)"
validation:
schema_check: true
keyword_match_threshold: 0.7
# -------------------------------------------------------------------------
# Edge Cases
# -------------------------------------------------------------------------
- id: tc015_no_contracts
description: "Skill handles case with no contracts gracefully"
category: edge_cases
priority: medium
input:
prompt: |
I have a new service with no consumers yet.
How should I set up contract testing?
context:
contractType: consumer-driven
expected_output:
must_contain:
- "consumer"
- "contract"
must_not_contain:
- "error"
- "impossible"
validation:
schema_check: true
allow_partial: true
- id: tc016_multiple_provider_versions
description: "Skill handles multiple provider versions correctly"
category: edge_cases
priority: medium
input:
prompt: |
Our provider has multiple versions in production:
- v1.x used by legacy consumers
- v2.x used by new consumers
How do we handle contract testing for both?
context:
multiVersion: true
expected_output:
must_contain:
- "version"
- "consumer"
- "provider"
must_match_regex:
- "(?i)(versioning|backward.*compatible|support)"
validation:
schema_check: true
timeout_ms: 60000
# =============================================================================
# Success Criteria
# =============================================================================
success_criteria:
# Minimum percentage of tests that must pass
pass_rate: 0.90
# Critical tests must have 100% pass rate
critical_pass_rate: 1.0
# Average reasoning quality across all tests
avg_reasoning_quality: 0.7
# Maximum time for entire suite (5 minutes)
max_execution_time_ms: 300000
# Maximum variance between different models (15%)
cross_model_variance: 0.15
# =============================================================================
# Metadata
# =============================================================================
metadata:
author: "@agentic-qe"
created: "2026-02-02"
last_updated: "2026-02-02"
coverage_target: >
Contract testing patterns including consumer-driven contracts (Pact),
provider verification, breaking change detection, semantic versioning,
canIDeploy decisions, Spring Cloud Contract, and mock generation.
Tests 16 scenarios across 9 categories.
adr_reference: "ADR-056"
trust_tier: 3
Contract Testing — Agent CLI Commands & Advanced Patterns
Merged from qe-contract-testing. Use these when working with v3 agent-specific contract capabilities.
AQE CLI Commands
# Generate contract from API spec
aqe contract generate --api openapi.yaml --output contracts/
# Verify provider against contracts
aqe contract verify --provider http://localhost:3000 --contracts contracts/
# Check breaking changes between versions
aqe contract breaking --old api-v1.yaml --new api-v2.yaml
# Test GraphQL schema
aqe contract graphql --schema schema.graphql --operations queries/Agent Workflow
// Contract generation
Task("Generate API contracts", `
Analyze the REST API and generate consumer contracts:
- Parse OpenAPI specification
- Identify critical endpoints
- Generate Pact contracts
- Include example requests/responses
Output to contracts/ directory.
`, "qe-api-contract")
// Breaking change detection
Task("Check API compatibility", `
Compare API v2.0 against v1.0:
- Detect removed endpoints
- Check parameter changes
- Verify response schema changes
- Identify deprecations
Report breaking vs non-breaking changes.
`, "qe-api-compatibility")GraphQL Contract Testing
await graphqlTester.testContracts({
schema: 'schema.graphql',
operations: 'queries/**/*.graphql',
validation: {
queryValidity: true,
responseShapes: true,
nullability: true,
deprecations: true
}
});Event Contract Testing
await contractTester.eventContracts({
schema: 'events/schemas/',
events: {
'user.created': {
schema: 'UserCreatedEvent.json',
examples: ['examples/user-created.json']
},
'order.completed': {
schema: 'OrderCompletedEvent.json',
examples: ['examples/order-completed.json']
}
},
compatibility: 'backward'
});Contract Report Interface
interface ContractReport {
summary: { contracts: number; passed: number; failed: number; warnings: number };
consumers: { name: string; contracts: ContractResult[]; compatibility: 'compatible' | 'breaking' | 'unknown' }[];
breakingChanges: { type: string; location: string; description: string; impact: 'high' | 'medium' | 'low'; migration: string }[];
deprecations: { item: string; deprecatedIn: string; removeIn: string; replacement: string }[];
}Pact Broker Integration
await contractTester.withBroker({
brokerUrl: 'https://pact-broker.example.com',
auth: { token: process.env.PACT_TOKEN },
operations: { publish: true, canIDeploy: true, webhooks: true }
});Coordination
Primary Agents: qe-api-contract, qe-api-compatibility, qe-graphql-tester Coordinator: qe-contract-coordinator
Provider States Reference
Provider states define the preconditions that must exist on the provider for a contract interaction to succeed.
Pattern: State Setup
// provider-states.js
const states = {
'a user exists': async () => {
await db.users.create({ id: 1, name: 'Test User', email: 'test@example.com' });
},
'no users exist': async () => {
await db.users.deleteAll();
},
'user 1 has 3 orders': async () => {
await db.users.create({ id: 1, name: 'Test User' });
await db.orders.bulkCreate([
{ userId: 1, status: 'shipped' },
{ userId: 1, status: 'pending' },
{ userId: 1, status: 'delivered' }
]);
}
};Common Mistakes
1. State not cleaned up — use transactions or truncate after each test 2. Hardcoded IDs — use factories or fixtures that generate consistent IDs 3. Missing state — provider test passes without state setup (no data = no error) 4. Overly specific states — "user John with email john@..." couples to consumer
Pact Broker Webhook Setup
# Verify provider when consumer publishes new contract
curl -X POST ${PACT_BROKER_URL}/webhooks \
-H "Content-Type: application/json" \
-d '{
"events": [{ "name": "contract_content_changed" }],
"request": {
"method": "POST",
"url": "${CI_TRIGGER_URL}",
"body": { "pact_url": "${pactbroker.pactUrl}" }
}
}'{
"$schema": "http://json-schema.org/draft-07/schema#",
"$id": "https://agentic-qe.dev/schemas/skills/contract-testing/output.json",
"title": "Contract Testing Skill Output Schema",
"description": "Schema for contract testing skill output with consumer-driven contracts, provider verification, breaking change detection, and Pact/Spring Cloud Contract support.",
"type": "object",
"required": ["skillName", "version", "timestamp", "status", "trustTier", "output"],
"properties": {
"skillName": {
"type": "string",
"const": "contract-testing"
},
"version": {
"type": "string",
"pattern": "^\\d+\\.\\d+\\.\\d+(-[a-zA-Z0-9]+)?$"
},
"timestamp": {
"type": "string"
},
"status": {
"type": "string",
"enum": ["success", "partial", "failed", "skipped"]
},
"trustTier": {
"type": "integer",
"minimum": 0,
"maximum": 3
},
"output": {
"type": "object",
"required": ["summary", "contractType", "verificationResult"],
"properties": {
"summary": {
"type": "string",
"minLength": 10,
"maxLength": 2000,
"description": "Human-readable summary of contract testing analysis"
},
"contractType": {
"type": "string",
"enum": ["consumer-driven", "provider", "bidirectional", "schema-first"],
"description": "Type of contract testing approach"
},
"framework": {
"type": "string",
"enum": ["pact", "spring-cloud-contract", "prism", "openapi", "graphql-schema", "custom"],
"description": "Contract testing framework used"
},
"verificationResult": {
"$ref": "#/definitions/verificationResult"
},
"consumers": {
"type": "array",
"items": {
"$ref": "#/definitions/consumer"
},
"maxItems": 100
},
"providers": {
"type": "array",
"items": {
"$ref": "#/definitions/provider"
},
"maxItems": 100
},
"contracts": {
"type": "array",
"items": {
"$ref": "#/definitions/contract"
},
"maxItems": 500
},
"interactions": {
"type": "array",
"items": {
"$ref": "#/definitions/interaction"
},
"maxItems": 1000
},
"breakingChanges": {
"type": "array",
"items": {
"$ref": "#/definitions/breakingChange"
},
"maxItems": 200
},
"canIDeploy": {
"type": "boolean"
},
"versionRecommendation": {
"$ref": "#/definitions/versionRecommendation"
},
"pactFiles": {
"type": "array",
"items": {
"$ref": "#/definitions/pactFile"
},
"maxItems": 100
},
"springCloudContracts": {
"type": "array",
"items": {
"$ref": "#/definitions/springCloudContract"
},
"maxItems": 100
},
"mockConfigurations": {
"type": "array",
"items": {
"$ref": "#/definitions/mockConfiguration"
},
"maxItems": 100
},
"findings": {
"type": "array",
"items": {
"$ref": "#/definitions/contractFinding"
},
"maxItems": 200
},
"recommendations": {
"type": "array",
"items": {
"$ref": "#/definitions/recommendation"
},
"maxItems": 50
},
"metrics": {
"$ref": "#/definitions/contractMetrics"
},
"categories": {
"type": "object",
"additionalProperties": {
"$ref": "#/definitions/categoryScore"
}
}
}
},
"metadata": {
"type": "object",
"properties": {
"executionTimeMs": {
"type": "integer",
"minimum": 0
},
"toolsUsed": {
"type": "array",
"items": { "type": "string" }
},
"agentId": {
"type": "string"
},
"pactBrokerUrl": {
"type": "string"
},
"contractVersion": {
"type": "string"
},
"environment": {
"type": "string",
"enum": ["development", "staging", "production", "ci"]
},
"targetBranch": {
"type": "string"
}
}
},
"validation": {
"type": "object",
"properties": {
"schemaValid": { "type": "boolean" },
"contentValid": { "type": "boolean" },
"confidence": {
"type": "number",
"minimum": 0,
"maximum": 1
}
}
},
"learning": {
"type": "object",
"properties": {
"patternsDetected": {
"type": "array",
"items": { "type": "string" }
},
"reward": {
"type": "number",
"minimum": 0,
"maximum": 1
}
}
}
},
"definitions": {
"verificationResult": {
"type": "object",
"required": ["status", "totalInteractions", "passedInteractions"],
"properties": {
"status": {
"type": "string",
"enum": ["passed", "failed", "pending", "partial"]
},
"totalInteractions": {
"type": "integer",
"minimum": 0
},
"passedInteractions": {
"type": "integer",
"minimum": 0
},
"failedInteractions": {
"type": "integer",
"minimum": 0
},
"pendingInteractions": {
"type": "integer",
"minimum": 0
},
"successRate": {
"type": "number",
"minimum": 0,
"maximum": 100
},
"failures": {
"type": "array",
"items": {
"$ref": "#/definitions/verificationFailure"
}
}
}
},
"verificationFailure": {
"type": "object",
"required": ["interaction", "reason"],
"properties": {
"interaction": { "type": "string" },
"reason": { "type": "string" },
"expected": { "type": "object" },
"actual": { "type": "object" },
"diff": { "type": "string" }
}
},
"consumer": {
"type": "object",
"required": ["name"],
"properties": {
"name": { "type": "string" },
"version": { "type": "string" },
"contractVersion": { "type": "string" },
"branch": { "type": "string" },
"environment": { "type": "string" },
"verificationStatus": {
"type": "string",
"enum": ["passed", "failed", "pending", "unknown"]
},
"lastVerified": { "type": "string" }
}
},
"provider": {
"type": "object",
"required": ["name"],
"properties": {
"name": { "type": "string" },
"version": { "type": "string" },
"branch": { "type": "string" },
"environment": { "type": "string" },
"verificationStatus": {
"type": "string",
"enum": ["passed", "failed", "pending"]
},
"consumerContracts": {
"type": "array",
"items": {
"type": "object",
"properties": {
"consumer": { "type": "string" },
"status": {
"type": "string",
"enum": ["passed", "failed", "pending"]
}
}
}
}
}
},
"contract": {
"type": "object",
"required": ["consumer", "provider"],
"properties": {
"consumer": { "type": "string" },
"provider": { "type": "string" },
"version": { "type": "string" },
"status": {
"type": "string",
"enum": ["verified", "failed", "pending", "stale"]
},
"interactions": {
"type": "array",
"items": {
"$ref": "#/definitions/interaction"
}
},
"providerStates": {
"type": "array",
"items": {
"$ref": "#/definitions/providerState"
}
},
"metadata": { "type": "object" }
}
},
"interaction": {
"type": "object",
"required": ["description"],
"properties": {
"description": { "type": "string" },
"providerState": { "type": "string" },
"providerStates": {
"type": "array",
"items": {
"$ref": "#/definitions/providerState"
}
},
"request": {
"$ref": "#/definitions/request"
},
"response": {
"$ref": "#/definitions/response"
},
"status": {
"type": "string",
"enum": ["passed", "failed", "pending"]
},
"failureReason": { "type": "string" }
}
},
"providerState": {
"type": "object",
"required": ["name"],
"properties": {
"name": { "type": "string" },
"params": { "type": "object" }
}
},
"request": {
"type": "object",
"properties": {
"method": {
"type": "string",
"enum": ["GET", "POST", "PUT", "PATCH", "DELETE", "OPTIONS", "HEAD"]
},
"path": { "type": "string" },
"query": {},
"headers": { "type": "object" },
"body": {},
"matchingRules": {
"$ref": "#/definitions/matchingRules"
}
}
},
"response": {
"type": "object",
"properties": {
"status": {
"type": "integer",
"minimum": 100,
"maximum": 599
},
"headers": { "type": "object" },
"body": {},
"matchingRules": {
"$ref": "#/definitions/matchingRules"
},
"generators": { "type": "object" }
}
},
"matchingRules": {
"type": "object",
"properties": {
"body": { "type": "object" },
"header": { "type": "object" },
"path": { "type": "object" },
"query": { "type": "object" }
}
},
"breakingChange": {
"type": "object",
"required": ["type", "description", "severity"],
"properties": {
"type": {
"type": "string",
"enum": [
"removed-endpoint",
"removed-field",
"type-change",
"required-field-added",
"response-change",
"status-code-change",
"removed-enum-value",
"narrowed-type",
"removed-parameter",
"changed-path"
]
},
"description": {
"type": "string",
"maxLength": 1000
},
"severity": {
"type": "string",
"enum": ["critical", "high", "medium", "low"]
},
"location": {
"type": "object",
"properties": {
"endpoint": { "type": "string" },
"method": { "type": "string" },
"field": { "type": "string" },
"path": { "type": "string" }
}
},
"affectedConsumers": {
"type": "array",
"items": { "type": "string" }
},
"migrationPath": {
"type": "string",
"maxLength": 2000
},
"deprecationDate": { "type": "string" }
}
},
"versionRecommendation": {
"type": "object",
"required": ["currentVersion", "recommendedVersion", "changeType"],
"properties": {
"currentVersion": { "type": "string" },
"recommendedVersion": { "type": "string" },
"changeType": {
"type": "string",
"enum": ["major", "minor", "patch", "none"]
},
"reason": { "type": "string" },
"breakingChangeCount": {
"type": "integer",
"minimum": 0
}
}
},
"pactFile": {
"type": "object",
"required": ["consumer", "provider"],
"properties": {
"consumer": {
"type": "object",
"required": ["name"],
"properties": {
"name": { "type": "string" }
}
},
"provider": {
"type": "object",
"required": ["name"],
"properties": {
"name": { "type": "string" }
}
},
"interactions": {
"type": "array",
"items": {
"$ref": "#/definitions/interaction"
}
},
"metadata": { "type": "object" },
"filePath": { "type": "string" }
}
},
"springCloudContract": {
"type": "object",
"properties": {
"name": { "type": "string" },
"description": { "type": "string" },
"priority": { "type": "integer" },
"ignored": { "type": "boolean" },
"request": { "type": "object" },
"response": { "type": "object" },
"filePath": { "type": "string" }
}
},
"mockConfiguration": {
"type": "object",
"required": ["name", "type"],
"properties": {
"name": { "type": "string" },
"type": {
"type": "string",
"enum": ["wiremock", "pact-stub", "spring-cloud", "msw", "nock", "prism"]
},
"endpoint": {
"type": "object",
"properties": {
"method": { "type": "string" },
"path": { "type": "string" }
}
},
"responses": {
"type": "array",
"items": {
"type": "object",
"properties": {
"statusCode": { "type": "integer" },
"body": {},
"headers": { "type": "object" },
"scenario": { "type": "string" }
}
}
},
"filePath": { "type": "string" }
}
},
"contractFinding": {
"type": "object",
"required": ["id", "title", "severity", "category"],
"properties": {
"id": {
"type": "string",
"pattern": "^CT-\\d{3,6}$"
},
"title": {
"type": "string",
"minLength": 5,
"maxLength": 200
},
"description": {
"type": "string",
"maxLength": 2000
},
"severity": {
"type": "string",
"enum": ["critical", "high", "medium", "low", "info"]
},
"category": {
"type": "string",
"enum": [
"contract-violation",
"missing-contract",
"stale-contract",
"breaking-change",
"coverage-gap",
"provider-state-missing",
"matcher-mismatch",
"schema-drift"
]
},
"consumer": { "type": "string" },
"provider": { "type": "string" },
"interaction": { "type": "string" },
"remediation": { "type": "string" }
}
},
"recommendation": {
"type": "object",
"required": ["id", "title", "priority"],
"properties": {
"id": {
"type": "string",
"pattern": "^REC-\\d{3,6}$"
},
"title": {
"type": "string",
"maxLength": 200
},
"description": {
"type": "string",
"maxLength": 2000
},
"priority": {
"type": "string",
"enum": ["critical", "high", "medium", "low"]
},
"effort": {
"type": "string",
"enum": ["trivial", "low", "medium", "high", "major"]
},
"category": {
"type": "string",
"enum": ["contract-coverage", "breaking-change-prevention", "versioning", "testing-strategy", "tooling"]
},
"codeExample": {
"type": "string",
"maxLength": 5000
}
}
},
"contractMetrics": {
"type": "object",
"properties": {
"totalContracts": { "type": "integer", "minimum": 0 },
"verifiedContracts": { "type": "integer", "minimum": 0 },
"failedContracts": { "type": "integer", "minimum": 0 },
"totalInteractions": { "type": "integer", "minimum": 0 },
"passedInteractions": { "type": "integer", "minimum": 0 },
"failedInteractions": { "type": "integer", "minimum": 0 },
"consumers": { "type": "integer", "minimum": 0 },
"providers": { "type": "integer", "minimum": 0 },
"breakingChanges": { "type": "integer", "minimum": 0 },
"coveragePercentage": {
"type": "number",
"minimum": 0,
"maximum": 100
},
"duration": { "type": "integer", "minimum": 0 }
}
},
"categoryScore": {
"type": "object",
"required": ["score"],
"properties": {
"score": {
"type": "number",
"minimum": 0,
"maximum": 100
},
"weight": {
"type": "number",
"minimum": 0,
"maximum": 1
},
"description": { "type": "string" },
"grade": {
"type": "string",
"pattern": "^[A-F][+-]?$"
}
}
}
}
}
{
"skillName": "contract-testing",
"skillVersion": "1.0.0",
"requiredTools": [
"jq"
],
"optionalTools": [
"pact",
"spring-cloud-contract",
"node",
"ajv",
"jsonschema",
"python3"
],
"schemaPath": "schemas/output.json",
"requiredFields": [
"skillName",
"status",
"output",
"output.contractType",
"output.verificationResult"
],
"requiredNonEmptyFields": [
"output.summary",
"output.contractType"
],
"mustContainTerms": [
"contract",
"consumer",
"provider"
],
"mustNotContainTerms": [
"TODO",
"FIXME",
"placeholder",
"example.com",
"lorem ipsum"
],
"enumValidations": {
".status": [
"success",
"partial",
"failed",
"skipped"
],
".output.contractType": [
"consumer-driven",
"provider",
"bidirectional",
"schema-first"
],
".output.verificationResult.status": [
"passed",
"failed",
"pending",
"partial"
]
}
}
Related skills
FAQ
What does contract-testing do?
contract-testing is a Claude Code skill for testing & qa.
When should I use contract-testing?
When you need to helps with testing & qa tasks., or when contract-testing is a claude code skill for testing & qa.
What are the main capabilities?
contract-testing; Testing & QA; AI-coding skill.